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Abstract 

The Renyi entropy with a free Renyi parameter q is the most justified form of 
information entropy, and the Tsallis entropy may be regarded as a linear approxi- 
mation to the Renyi entropy when q ~ 1 . When q — ► 1 , both entropies go to the 
Boltzmann-Shannon entropy. The application of the principle of maximum of infor- 
mation entropy (MEP) to the Renyi entropy gives rise to the microcanonical (homoge- 
neous) distribution for an isolated system. Whatever the value of the Renyi parameter 
q is, in this case the Renyi entropy becomes the Boltzmann entropy Sb = kslnW , 
that provides support for universality of the Boltzmann's principle of statistical me- 
chanics. For a system being in contact with a heat bath, the application of MEP to 
the Renyi entropy gives rise to Levy distribution (or, q -distribution) accepted as one 
of the main results of the so-called nonextensive statistics. The same distribution is 
derived here for a small physical system experiencing temperature fluctuations. The 
long-range "tail" of the Levy distribution is the power-law (Zipf-Pareto) distribution 
with the exponent s expressed via q . The exponent and free Renyi parameter q 
can be uniquely determined with the use of a further extension of MEP. Then typical 
values of s are found within the range 1.3-^2 and of q within the range 0.25-^0.5, 
in dependence on parameters of stochastic systems. 
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1 Introduction 

In view of the rapid development of non-extensive statistics based on Tsallis' information 
entropy the problem of choosing the form of information entropy becomes pressing. The 
most justified one appears to be one-parameter family of Renyi's entropies (or simply Renyi 
entropy). When Renyi parameter q is equal to unity, Renyi entropy goes to well-known 
Boltzmann-Shannon entropy. The Renyi entropy is extensive but its linearization in the 
neighbourhood of a point q ~ 1 results in the non-extensive Tsallis' entropy. 

When the principle of maximum of an information entropy (MEP) is applied to the 
Renyi entropy of an isolated system, a homogeneous (microcanonical) distribution pi = 
1/W over W micro-states is derived. In this case, the Renyi entropy becomes the Boltz- 
mann entropy Sb = ks^W thus afforcing Boltzmann Principle from which all thermo- 
dynamic properties of extensive and non-extensive Hamiltonian systems can be deduced. 

For a system in contact with a heat bath, the Levy distribution (or q -distribution) 
{pi} over microscopic states of a system is derived with the use of MEP applied to the 
Renyi entropy. Applying this principle to the Tsallis entropy one would be forced to go 
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from the original distribution {pi} to an escort distribution {Pi} = {pJ/J2iPi} , an d use 
it further for evaluations of mean values of dynamical variables. Such going contradicts to 
main principles of probability description. 

The Levy distribution can be obtained also for small subsystems experiencing large 
fluctuations of temperature by way of averaging of Gibbs distribution over the temperature 
fluctuations. Besides, this procedure permits to associate the Renyi parameter q with 
physical properties of the subsystem. 

A long-range "tail" (for % >> 1 ) of the Levy distribution can be approximated by a 
power-law distribution pi ~ x^ s , where the exponent s is expressed in terms of q . Then 
at least for power-law distributions, the MEP can be expanded to determine the Renyi 
parameter q and the exponent s , respectively. 

2 Information entropy 

The information entropy, or simply entropy, is the measure of uncertainty in information 
in the case of statistical (incomplete) description of a system using the distribution of 
probabilities p = {pi}, < pi < 1, i — 1, W. The best known is the representation of 
entropy in the Boltzmann-Shannon form 

w 

S B = -k B ^pi\npi. (1) 

i 

In the case when the subscripts i indicate dynamic microstates in the Gibbs phase space 
and the distribution Pi corresponds to the macroscopic equilibrium state of the system, 
the entropy Sb coincides with the thermodynamic entropy. 

It is just this type of entropy was justified by Khinchin [1] and Shannon [2] in a theorem 
form based on a system of axioms. Their axioms were analyzed in [3,4], where it was shown 
that a unique determination of entropy in the Boltzmann-Shannon form is provided by 
a quite artificial axiom related to a form of conditional entropy (that is, the entropy of 
subsystem of a system being in a prescribed state). A variety of papers on this subject 
were analyzed by Uffink [4] who found that the most convincing appears to be the system 
of axioms of Shore and Johnson [5] leading to Renyi's one-parameter family of entropies 
[6]; for the distribution {p^} normalized to unity, this family is written in the form 

h. w w 

4 9) (p) = -r^-inE^ Eft = 1 > ( 2 ) 

" i i 

where q is an arbitrary positive number (it cannot be less than zero, because {p^} may 
include zero values). Various properties of Renyi entropy are discussed, in particular, in 
the monographs [6-8]. Among its basic properties we may mention: positivity ( Sr > ), 
concavity for q < 1 and, in addition, lim^x Sr = Sb ■ 

In the case of |1 — J2iPi \ ^ 1 (which, in view of normalization of the distribution {p^} , 
corresponds to the condition |1 — q\ <C 1 ), one can restrict oneself to the linear term of 
logarithm expansion in the expression for Sft\p) over this difference, and S^\p) changes 
to 

h w 

4 9) (p) = -y^-(i-E^). (3) 
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Such a linearization of Renyi entropy was first suggested by Havdra and Charvat [9] and 
Daroczy [10]; at present, this expression for entropy came to be known as Tsallis' entropy 

The logarithm linearization results in the entropy becoming nonextensive. This property 
is widely used by Tsallis and by the international scientific school that has developed around 
him for the investigation of diverse nonextensive systems [11-16]. In so doing, the above- 
identified restriction |1 — q\ <C 1 is disregarded, as a rule. In our opinion, the attempt by 
Abe [16] at independent validation of this form of entropy appears unconvincing, because 
it is based on the axiomatic introduction of such a form for conditional entropy which 
uniquely provides for obtaining Tsallis' entropy. 



3 Extremality of entropy 

According to MEP, in the case of statistical (incomplete, from the dynamic standpoint) 
description of the system, its distribution function must provide for correct values of those 
few average quantities which appear in the statistical description; otherwise, it must be as 
undetermined as possible. Such an approach in application to equilibrium thermodynamic 
systems (isolated or weakly interacting with the thermostat) has long been used to construct 
equilibrium statistical thermodynamics. However, it was only after studies by Jeynes [17] 
that it came to be firmly established as a principle validating (at least, on the physical level 
of rigor) the use of Gibbs ensembles in statistical description of thermodynamic systems. 
The information entropy is traditionally taken to mean the Boltzmann-Shannon entropy. 
Here, MEP will be applied to the Renyi entropy. 



3.1 Microcanonical distribution. Boltzmann principle. 

We are interested firstly in the distribution of probabilities {pi} , providing for the ex- 
tremality of information entropy with an additional condition of normalization of pi . 
Then, the distribution {p^ must be determined from the extremum of the functional 

i w w 

^) = — ln E^-^Eft» (4) 

" i i 

where $ is the Lagrange multiplier dependent on q . 
We equate its functional derivative to zero to derive 

MM = ^-* = », ( 5) 

Multiplying this equation by pi and summing up over % , with account of normalization 
condition J2iPi = 1 we get $ = • Then, it follows from equation (4) that 

(w \^ 

Using once more the condition J2iPi — 1 we get 

/f 1 
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and, finally, 

Thus, we have obtained equally probable distribution pi corresponding to the microcanon- 
ical Gibbs ensemble of statistical mechanics. The Renyi entropy for this case takes a form 

S M = kB_ ^ W l-g = k ^ W 

1 - q 

This expression does not depend on the Renyi parameter and at any q coincides exactly 
with the Boltzmann entropy definition Sb = ks^W called by Einstein as Boltzmann's 
Principle from which all equilibrium statistical thermodynamics both extensive and non- 
extensive Hamiltonian systems can be deduced (see, e.g. [18]). It was proposed [18] to take 
Boltzmann Principle as the axiomatic assumption. Here we obtained it as a consequence 
of the axiomatics of Shore and Johnson [5] led us to Renyi entropy and then to Boltzmann 
Principle. 

Thus, the Renyi entropy gives new physical insight into the Boltzmann Principle. The 
Tsallis entropy exhibits this mandatory property only when q — > 1 . 

3.2 Canonical Levy distribution. 

Next we will look for the distribution of probabilities {p^} , providing for the extremality 
of information entropy with an additional conditions which consist in preassigning the 
average value H = (H) p = J2i HiPi of the random quantity ifj and the requirement of 
normalization of pi . 

Then, the distribution {p^ must be determined from the extremum of the functional 

i w w w 

L R {p) = — In £ p\ - (3 q £ H iPi (8) 

9 i i i 

where f3 Q q and $ are Lagrange multipliers dependent on q . Note that, in the q — > 1 
limit it changes to well-known functional 

w WW 

Lg{p) = -J2P^ n Pi-PoJ2 H iPi - $ Yl Vi\ (9) 

i i i 

its extremum is ensured by the Gibbs canonical distribution, in which (3$ = l/ksT and 
$o is the free energy. 

We equate a functional derivative of Lr(p) to zero to derive 

%m = ^|t;_ a =0 (10) 

OPi l-q Ej Pj 

Multiplying this equation by Pi and summing up over i , with account of normalization 
condition Yl,iPi — 1 we get $ = jz^ — PoqH . Then, it follows from equation (10) that 

i 

(W \ -J-i 

5>?(1 + #,(1- if)) 



4 



Using once more the condition J2iPi — 1 we get 



w 




"(9-1) 



9-1 



and, finally, 



p« = — ^- rr^- ( n ) 

^(l + A^i-sX^-tf))"- 1 

Such distribution is known now as Levy distribution or q -distribution. At q — > 1 the 
distribution {pi} becomes the Gibbs canonical distribution in which the constant (5q = 
I/UbTq is the reciprocal of the temperature. 

If the Tsallis entropy was used instead of the Renyi entropy the Levy distribution was 
derived [11] in the form 

Ts (1 + /? (1 

£i(i + A)(i-<z)ifi 

but the starting functional was forced to be taken as 



p T S = v- '^ (12) 

9-1 



Q \ i / i i 



(13) 



Here, the question arises about forms of Lagrange multipliers $@o(q — 1) and (+$) , but 
the main problem is that the functional L Ts {p) does not pass to the functional (9) when 
q — > 1 as the second term in (13) vanishes. 

It seems reasonable to suppose that just this difficulty causes to use the escort distri- 
bution Pi=Pi/J2iPi in nonextensive thermodynamics [12,13,15]. The consistency of the 
transition to escort distribution is partly justified by the condition conservation of a preas- 
signed average value of the energy H = (H) es = J2i Hi Pi , however other average values 
are to be calculated with the use of the same escort distribution also, that contradicts to 
the main principles of probability description. 

It should be noted also that escort distributions were introduced [7] as a tool to scan 
the structure of an original distributions {pi} . Indeed, at q > 1 , the importance of pi 
with the maximal values increases, and at q < 1 , of pi with minimal values. In view of 
this, it is evident that use of escort distributions in statistical thermodynamics does not 
lead to true average values of dynamical variables. 



4 Small subsystem with fluctuating temperature 

To clear up a physical sense of the Levy distribution (11) and the Renyi parameter q we 
use here an approach proposed by Wilk and Wlodarchuk [19]. 

We will treat a subsystem which is a minor part of a large equilibrium system and 
experiences thermal fluctuations of both energy and temperature. This is a radical difference 
of the suggested approach from the Gibbs approach traditionally employed in statistical 
physics, in which temperature is preassigned by a constant characterizing the thermostat. 

In equilibrium theory of thermodynamic fluctuations the mean square temperature fluc- 
tuation is estimated as 5T/T = (kB/C) 1 / 2 where C is the heat capacity of the subsystem. 
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More detailed analysis [20], based on the Landau-Lifshits theory of hydrodynamic fluctu- 
ations, gives the next nonlinear stochastic Langevin equation for fluctuating temperature 

^ = -^{T(t)-T )-±T(t)t(t), (14) 

where r = C/{Ak) , A is the surface area of the subsystem, k is the heat transfer 
coefficient, and T is the average temperature of the subsystem. A random function of 
time £(t) satisfies the relation 

(£(*)> = 0, mat')) = 2k B An5{t - f ). (15) 

Corresponding to the derived stochastic Langevin equation with 5 -correlated noise is 
the Fokker-Planck equation for the temperature distribution function f(T,t) 

df(T t) d Id 2 

J} 0f 1 = -^iCO/mO + - 2W2 W 2 (T)f(T,t). (16) 

The coefficients W\{T) and W^iT) of this equation are expressed in terms of the first 
(T(t) — T(t + r)) and second {(T(t) — T(t + r)) 2 ) conditional moments of stochastic 
equation Langevin. A steady-state solution [20,21] to equation (16) is 

m = ^-f- 1 ^- (it) 

where the dimensionless constant 7 = C/ks is introduced, and z = r yT /T . Therefore, 
the thus derived distribution function of the inverse temperature of the subsystem in the 
dimensionless form is the gamma distribution. 

If the mean energy of the singled-out volume H = CT is introduced, this expression 
may be rewritten as 

™ = {J $iw'~' >B - (18) 

By its form, this distribution is close to the Gibbs distribution; however, unlike the latter, 
it accounts for the temperature fluctuation of the subsystem with the preassigned mean 
energy H . 

In order to describe a subsystem in contact with a large thermally equilibrium system 
(thermostat), the Gibbs canonical distribution is used in statistical physics (here and below, 
the factor Gj allowing for number of states of energy H \ is omitted for brevity) : 

Pl = Q-ie-™, (19) 

where Hi is the energy of the subsystem (the subscript % may indicate the number of 
discrete energy level or totality of the values of coordinates and momenta of molecules of 
the subsystem), and Q is the partition function. In so doing, the inverse temperature 
(3 = 1/ksT is taken to be known preassigned quantity. 

As was demonstrated above, the temperature may fluctuate. In view of this, the ques- 
tion arises as to how the Gibbs distribution is modified under the effect of temperature 
fluctuations. The answer to this question may be obtained by the way of averaging the 
Gibbs distribution (19) with the gamma-distribution for temperature T (or (5). 

For further treatment, pi may be conveniently represented in an equivalent form, 

fH = Q- 1 e- pAHi , Q = £e-^, AH i = H i -H (20) 
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where the symbol J2i may indicate both the summation and integration over a totality of 
the values of coordinates and momenta. 

Using the mean value theorem we represent the Gibbs distribution averaged over (3 in 
the form 

/*oo 1 roc 

p t = j Q dPfifl/H = —j o d(5f{(5)e-^ H % (21) 

where Q* lies in the range of possible variation of Q(/3) from Q(0) to Q(oo) . From the 
conditions of normalization to unity of the distributions f{(5) and p , we have 

I r°° 

^E/ dMfle-f 1 ** = 1 (22) 

whence we find ^ 

Q* = Y, dPfifte-^. (23) 

Therefore, it is sufficient to calculate only the average value of the exponent, 

nd(3f([3)e^ AH > = ^^V - 1 rW 1 ^ 7 ™' = (I + -^Hy\ (24) 
Jo T(7) Jo v 7 ' 

Finally, the averaged Gibbs distribution takes the form 

In the 7 — > oo limit corresponding to a high heat capacity of the singled-out subsystem, 
Pi goes to pi . 

Resulted equation for the modified Gibbs distribution is similar to equation (11) for pi 
in its structure. To identify pi with Pi it is enough to present 7 as 7 = (1 — g) _1 , then 
equation (25) takes the form 

Pi = — K — 7 ^ . (26) 

Ei(l+Po(l-q)(Hi-H)) 

The full identity of this expression with the probability Pi (11) ensuring the extremality of 
Renyi entropy enables one to take a new view of the physical meaning of the Renyi entropy 
and parameter 

So, the Renyi parameter differs significantly from unity only in the case where the heat 
capacity of the singled out system is of the same order of magnitude as the Boltzmann 
constant ks ■ The thermodynamics of such systems must be constructed on the basis of 
Renyi entropy and Boltzmann Principle or the Levy distribution function {pi] or {p^ for 
systems in contact with a heat bath. We shall emphasize once again that here this function 
was obtained without invoking any additional considerations as to the nonextensiveness of 
the systems being treated. 
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When % -th state of the system is determined by a value Hi (indexed in increasing order 
of magnitude, so that H max = H w ), and Xi = Hi/H w then for q ^ 1 and sufficiently 
significant fluctuations of x exceeding the minimal value 



x <> 



> 



1 



X 



P H w (l-q) 



expression (6) transforms to the power distribution 



1 



Pi 



(28) 



(29) 



So, we derived the Zipf-Pareto power-law distribution ensuring the extremality of Renyi 
entropy at q ^ 1 . In this aspect it can be said that the power-law distribution is inherent 
to the Renyi entropy as much as Boltzmann or Gibbs distributions are inherent to the 
Boltzmann-Shannon entropy. As the exponent s is expressed (29) in terms of the Renyi 
parameter q , the evident requirement s > produces a new constraint q < 1 that 
coincides with the condition of concavity of the Renyi entropy 1 . Otherwise the Renyi 
parameter remains arbitrary, so while it is varying from q <C 1 to q ~ 1 the exponent s 
varies from 1 to +oo . Hence it is evident that an additional physical concept should be 
invoked to specify q uniquely. 



5 Extension of MEP to determine Renyi parameter 

As a rule, the exponent s of power-law distributions for stochastic systems of different 
nature lies in the narrow range of values between 1 and 2. It follows from it that a 
corresponding value of the Renyi parameter q connected with s by equation (29) lies too 
in a narrow range. 

In this connection, it is reasonable to assume an existence of some variational principle 
providing realization of value of the Renyi parameter just in this range. Inasmuch as the 
Levy distribution possessing the power-law "tail" is inherent in the Renyi entropy, it seems 
to be reasonable to look for extremum on q of the difference 

AS = S%\{ Pi }) - S B ({p t }) (30) 

where {pi} is the Levy distribution (11). 

When the subscript \Api\ = \pi+i — Pi\ ^ Pi for all i , we can pass to a continuous 
picture. Then the probability pi is replaced by p(xi)Axi , where p(x) is the probability 
density and Axi = x i+ \ — Xi . For simplicity, we assume that all Axi are of the same 
value ( Axi = Ax for all % ), then we get 

i w 

4 9) (M) = — lnJXsOCAs)* = -\nAx + S^ D (p(x)), (31) 

? i 

W 

S B ({Pi}) = -Y,p(x i )(Ax)hi\p(x i )(Ax)] = -\nAx + S BD (p(x)) (32) 

i 

1 When one tries to derive the power-law distribution on the base of the escort distribution he obtains 
the exponent s es = q/(q — 1) that produces a constraint q > 1 . For such q the Renyi entropy is nor 
pure concave. 
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Fig. 1: The difference AS = S$ — Sb for power-law distribution p ~ x x l x q as a 
function of q and x m i n . 

where ^ i i 

s rd(p( x )) = i ln / P q (x)dx, S BD (p(x)) = - p(x) \np(x)dx. (33) 

1 — (J J XI J XI 

It is suggested that the probability density and differential Arc satisfy all requirements 
which are necessary when passing from Darboux sums to integrals in equations (31) and 
(32). 

It is conventional that Sbd is referred to as the differential Boltzmann entropy, so 
we call S q RD differential Renyi entropy. The original Boltzmann and Renyi entropies are 
positive defined values, but differential ones are not positive defined due to smallness of 
Ax . Nevertheless, their difference is the same as the difference of original entropies, so 
that 

AS = S%Up(x))-Sbd(p(x)). (34) 

An investigation of this function of q for the presence of its extremum is only possible 
for a concrete system when all parameters, (3$, H of the Levy distribution are known. 
However, if a main contributions into the integrals of differential entropies (34) are provided 
by the range of x values from x m i n to x max = 1 , we can use the power-law distribution 
density instead of Levy distribution density 

p(x\s) = B^x- 3 , B= f 1 x~ s dx = — ^(1 - x 1 -^). (35) 

Then 

AS(q) = S^ D (p(x\s)) - S BD (p(x\s)) 

/; ' d; ";[(l-^n)« + ^(«-l)ln5 



(s-iy 

+ (l-s) HBx-JJ] + ^— - ln r Y_ g 7 m ) • (36) 



1 . (BH\-x\~ 



This function of q depends on the parameter x min only. Its three-dimensional plot is 
illustrated in Fig. 1. Well defined maximum with respect to q is seen at it and its 
position q* depends slightly on x m i n . Substituting q = q*(x m i n ) into equation (29), we 
get s* = 1/(1 — q*(r)) , which is illustrated in Fig. 2. It is seen that values of s*(x m i n ) 
are in the range from 1.1 to 2 depending on x m i n ■ 
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Fig. 2: The exponent of the power-law distribution s* = 1/(1 — q*) as a function of x min . 

Distributions of this kind were found for an extremely wide class of natural and social 
phenomena. As examples, we can mention the distributions of magnitudes of earthcrack- 
ings (Gutenberg-Richter law), vortices over their energies in turbulent flows, and also in 
the science of sciences [22], economics (Zipf-Pareto distribution law for citizens over their 
incoming or enterprizes over number of collaborators, or banks over their capitals), geog- 
raphy (distribution law for countries or cities over their citizens) etc. 

In these examples, we deal with parameters being analogs of energy. In opposite cases, 
when we are interested in the power-law distribution over other parameters, say I , we 
should recalculate the exponent. Really, if H ~ V then p(x\s)dx ~ H s dH ~ Z _rs+r_1 oW , 
that is, the modified exponent is s\ = rs — r + 1 . As an example, for the distribution 
of fragments over their masses in impact fragmentation of solids, we have [23] H ~ m 2 / 3 
and x m i n — 10~ 5 , then, from the Fig. 2, we find s ~ 1.5 whence s m ~ 1.3 , that is in 
agreement with the experimental results [24] and theoretical estimations [23]. 
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